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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address ■ 
Period for Reply 

_ MONTH(S) FROM 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE _ 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

• tf the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 
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- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

I) KI Responsive to communication(s) filed on 3 March 2004 . 

2a)M This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) S Claim(s) 1-38 is/are pending in the application. 
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5) D Claim(s) is/are allowed. 

6) M Claim(s) 1-38 is/are rejected. 
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Application Papers 
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Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
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2.D Certified copies of the priority documents have been received in Application No. . 
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application from the International Bureau (PCT Rule 17.2(a)). 
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DETAILED ACTION 



The amendment filed on 3 March 2004 have been received and entered. Claims 1-38 are 



pending in the application. 



Claim Rejections - 35 USC § 102 



The following is a quotation of the appropriate paragraphs of 35 U.S. C. 102 that form the 
basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

2. Claims 1-7, 9, 1 1-17, 19, 21-26, 28, 30-35 and 37 are rejected under 35 U.S.C 102(b) as 
being anticipated by the article entitled "Color SuperHistograms for Video Representation", 
written by Dimitrova et al. 

Referring to claims 1, 11,21 and 30, Dimitrova et al. teach an apparatus, system, method 
and computer executable instructions comprising a visual summary controller capable of creating 
a visual summary of video material (page 316, Figure 1), wherein the visual summary controller 
is capable of extracting frame signatures (histograms) from keyframes of video material and 
capable of using the frame signatures to create superhistograms from the keyframes, and capable 
of using the frame signatures and superhistograms to create a visual summary of the video 
material (the superhistograms created from the frame signatures represent a visual summary of 



Application/Control Number: 09/866,394 Page 3 

Art Unit: 2173 

the video content), as recited on page 314, right column, lines 1 1-25, page 315, section 2 and 
page 316, section 2.3; this is further shown in Figure 1. 

Referring to claims 2, 12, 22 and 31, Dimitrova et al. teach the filtering of keyframes 
(merging of histograms into family histograms) and extracting frames signatures (computing 
color histograms) from the filtered keyframes before using the frame signatures (histograms) to 
create the superhistogram representing a visual summary of the video material (page 315, right 
column, section 2 and page 316, left column, section 2.3). 

Referring to claims 3, 13, 23 and 32, Dimitrova et al. teach the use of superhistograms to 
cluster the filtered keyframes (the ordered merging of the family histograms to create the 
superhistogram), wherein the clustered keyframes (superhistogram) represents the visual 
summary of the video material, as recited on page 314, right column, lines 1 1-25 and shown in 
Figure 1. 

Referring to claims 4 and 14, Dimitrova et al. teach the use of a histogram as the frame 
signature used to compute superhistograms (page 314, right column, lines 11-15). 

Referring to claims 5, 15, 24 and 33, Dimitrova et al. the use of the LI distance measure 
method, L2 distance measure method, histogram intersection method, Chi-Square test and Bin- 
wise histogram intersection method to computer the histogram difference (page 315, right 
column). 

Referring to claims 6, 16, 25 and 34, Dimitrova et al. teach the selection of the most 
meaningful image for each superhistogram (the top n largest families) as the representative 
image (page 3 1 6, section 2.4). 
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Referring to claims 7, 17, 26 and 35, Dimitrova et al. teach the ability to select the family 
histograms (the top n largest families) to use to create the superhistogram used to create the 
visual summary (page 316, section 2.4). 

Referring to claims 9, 19, 28 and 37, Dimitrova et al. teach the use of the visual summary 
obtained from the superhistograms to access at least a portion of the video material (classifying 
and searching in video archives), as recited on page 317, section 4.2. 



Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S. C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

3. Claims 8, 10, 18, 20, 27, 29, 36 and 38 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over the article entitled "Color SuperHistograms for Video Representation", written 
by Dimitrova et al., in view of Wang et al. U.S. Patent 5,805,733. 

Referring to claims 8, 18, 27 and 36, while Dimitrova et al. teach all of the limitations as 
applied to the claims above, they fail to teach the capability to retrieve a visual summary stored 
in a memory unit and causing the visual summary to be displayed in response to a user request. 
Wang et al. teach the summarization of video sequences similar to that of Dimitrova et al. In 
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addition, Wang et al. further teach the capability of letting a user select a visual summary (scene) 
for viewing, retrieving that visual summary from memory and displaying it in response to the 
user's request, as recited in column 2, lines 16-29 and shown in Figure 2. It would have been 
obvious to one of ordinary skill in the art, having the teachings of Dimitrova et al. and Wang et 
al. at the time the invention was made, to modify the video summarization scheme disclosed by 
Dimitrova et al. to include the retrieval and display of the visual summary in response to a user 
request, as taught by Wang et al. One would have been motivated to make such a combination to 
give users the flexibility to select which scenes to watch, saving them time from having to 
browse through all of the other irrelevant scenes. 

Referring to claim 10, 20, 29 and 38, while Dimitrova et al. teach all of the limitations as 
applied to the claims above, they fail to teach the creation of new video material using the 
compact visual summaries. Wang et al. teach the creation of new video material (a collage) 
made up of representative frames for each compact visual summary, as recited in column 3, lines 
53-57. It would have been obvious to one of ordinary skill in the art, having the teachings of 
Dimitrova et al. and Wang et al. at the time the invention was made, to modify the video 
summarization scheme disclosed by Dimitrova et al. to include the creation of new video 
material, as taught by Wang et al. It would have been advantageous for one to utilize such a 
combination in order to conserve processor time and storage space by utilizing the already 
existing visual summaries in the creation of new visual materials. 



Response to Arguments 
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4. Applicant's arguments filed on 3 March 2004 have been fully considered but they are not 
persuasive. 

Applicant asserts that Dimitrova et al. do not disclose or suggest a controller which is 
capable of extracting frame signatures from keyframes, and capable of using the frame signatures 
to create superhistograms from the keyframes, and capable of using the frame signatures and the 
superhistograms to create a compact visual summary. However, it is disclosed in the Dimitrova 
et al. reference that the method for computing superhistograms is capable of extracting frame 
signatures (histograms) from keyframes of video material and capable of using the frame 
signatures to create superhistograms from the keyframes , and capable of using the frame 
signatures and superhistograms to create a visual summary of the video material (the 
superhistograms created from the frame signatures represent a visual summary of the video 
material, or scenes), as recited on page 314, right column, lines 1 1-25, page 315, section 2 and 
page 316, section 2.3; this is further shown in Figure 1. The Dimitrova et al. reference teaches 
extracting color histograms for individual shots (key frames) of video content. Histograms are a 
type of frame signature, as is even disclosed in the applicant's application, shown in the 
limitation recited in claim 4. Therefore, the extraction of color histograms is one way of 
extracting frame signatures from key frames of video material. The color histograms are used to 
create family histograms, which are used to create superhistograms. Therefore, the frame 
signatures are used to create superhistograms. The frames signatures and superhistograms are 
used to classify and analyze video data and is therefore a visual summary of the video material. 
The Dimitrova et al. reference thus anticipates the subject invention. 
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5. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 . 136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 .136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 



Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Ting Zhou whose telephone number is (703) 305-0328. The 
examiner can normally be reached on Monday - Friday 8:00 am - 5:30 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John Cabeca can be reached on (703) 308-3 1 16. The fax phone number for the 
organization where this application or proceeding is assigned is (703) 872-9306. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



March 22, 2004 



